1. Field of the Invention
The present invention relates to a multimedia information processing system in which a digitalized video signal, an audio signal and other data are coded, multiplexed, and then, transmitted or stored, and the transmitted signal or the stored signal are demultiplexed, and then, decoded to generate the video signal, audio signal and other data. As specific applications of the multimedia information processing system, there are a video on demand (VOD), a digital broadcasting system, and a video conference system.
2. Description of the Prior Art
FIG. 10 is a block diagram showing a structure of a video encoder (transmitter) forming a conventional multimedia information processing system described in "A Video Codec for Digital Satellite News gathering Systems", Mitsubishi Electric Corporation Technical Publication, Vol. 67, No. 7, pp 33-38, 1993.
In FIG. 10, reference numeral 91 is an information-source coding section which inputs a video signal 101, an audio signal 102, and the other data signal 103 and executes data compression for those signals. In the information-source coding section 91, a video coding portion 51 executes information-source coding of the video signal 101 to generate the video coded bits sequence 111. An audio coding portion 512 executes information-source coding of one or more audio signals 102 to generate one or more audio coded bits sequences 112. The data coding portion 513 executes various processings concerning the data signal 103 relating to the video signal 101 or the audio signal 102 to generate a data coded bits sequence 113.
A media multiplexing section 92 multiplexes the video coded bits sequence 111, the audio coded bits sequence 112 and the data coded bits sequence 113 from the information-source coding section 91 to generate one multiplexed bits sequence 122. A transmission processing section 93 provides the multiplexed bits sequence 122 with scramble processing and error correction coding in accordance with the types and characteristics of the transmission line. The transmission processing section 93 further executes channel multiplexing processing for the multiplexed bits sequence 122 and one or more other multiplexed bits sequences 123. Furthermore, the transmission processing section 93 executes the formation of a transmission frame and modulating processing and so on are executed to generate a transmission signal 141.
FIG. 11 is a block diagram showing a structure of a video decoder (receiver) forming the multimedia information processing system described in the above report. In FIG. 11, reference numeral 96 denotes a receive processing section which receives the transmission signal 141, executes demodulating processing, synchronization of the transmission frame, regeneration of the transmission frame and separation of the channel multiplexing processing to select a specific channel signal. The receive processing section 96 further executes error detecting processing, error correcting processing and descramble processing or the like to reproduce multiplexed bits sequence 122 or one or more other multiplexed bits sequences 123.
A media demultiplexing section 95 separates the multiplexed bits sequence 122 into the video coded bits sequence 111, one or more audio coded bits sequences 112 and one or more data coded bits sequences 113.
An information-source decoding section 94 inputs the video coded bits sequence 111, one or more audio coded bits sequences 112 and one or more data coded bits sequences 113 and also outputs the video signal 101, one or more audio signals 102 and one or more data signals 103. In the information-source decoding section 94, a video decoding portion 514 executes information-source decoding of the video coded bits sequence 111 to reproduce the video signal 101. An audio decoding portion 515 executes information-source decoding of the one or more audio coded bits sequences 112 to reproduce the audio signal 102 relating to the video signal 101. A data decoding portion 516 executes various reverse processings to the data coded bits sequence 113 to reproduce the data signal 103 relating to the video signal 101 and audio signal 102.
FIG. 13 is an explanatory diagram showing a general function of the transmission processing section 93 and the receive processing section 96. As shown in FIG. 13, the transmission processing section 93 inputs multiplexed bits sequences and produces transmission signals by performing a framing process, modulation process and physical layer process. The receive processing section 96 receives transmission signals and reproduces multiplexed bits sequences by performing physical layer process, demodulation process and deframing process.
Operations in the transmission side of the multimedia information processing system will now be described. The information-source coding section 91 reduces redundant components of the respective signals by use of information-source coding method. Information of the video signal 101 and the one or more audio signals 102 which are accompanied therewith are coded in a video coding portion 511 and audio coding portion 512 respectively. Information of the data signal 103 including synchronization signals and control signals relating to the video signal and audio signal are also coded in a data coding portion 513 as in the case of the video signal and audio signal, thereby converting the data structure to form one or more data coded bits sequences 113.
The media multiplexing section 92 multiplexes the video coded bits sequence 111, audio coded bits sequence 112 and data coded bits sequence 113 into one multiplexed bits sequence 122. The multiplexing is executed by use of a given frame as a unit. FIG. 12 is an explanatory view explaining multiplexed frame generated in the media multiplexing section 92. As shown in FIG. 12, a predetermined sized multiplexed frame 80 is composed of a synchronizing information area 81 for securing the synchronization of the multiplexed frame, a video information area 82, audio information area 83 and a data information area 84. The video coded bits sequence 111, the audio coded bits sequence 112 and the data coded bits sequence 113 are provided in the video information area 82, the audio information area 83, and the data information area 84 respectively and are read out in the order shown with arrow in FIG. 12 (from the left upper portion to the right lower portion), so that the multiplexed bits sequence 122 is generated. To multiplex signals using such a multiplexed frame as a unit is referred to as "structure multiplexing" in this specification.
The transmission processing section 93 executes scramble processing for the multiplexed bits sequence 122 and adds error correcting codes to the multiplexed bits sequence 122. Further, the transmission processing section 93 executes channel multiplexing for the multiplexed bits sequence 122 and other one or more multiplexed bits sequences 123 to generate a transmission frame. The transmission processing section 93 then executes modulation processing or the like for the transmission frame to generate a transmission signal 141.
The scramble processing in the transmission processing section 93 will be described using FIG. 14(a). FIG. 14(a) shows one example of a circuit that realize a scramble processing referred to as a PN (Pseudo Noise) signal summing system. In FIG. 14(a), reference numeral 73a denotes a pseudo random number generator for scramble and 75 an exclusive-OR circuit. The random number generator 73a generates a pseudo random number sequence 74a responsive to an initial value 72a. The exclusive-OR circuit 75 executes exclusive-OR operation of original data 71 and the pseudo random number sequence 74 a as a bit unit to obtain scrambled data 76.
A structure of the error correction frame in a case where the transmission processing section 93 executes error correction coding will be described. FIG. 15 is a block diagram of an error correction frame 77 in a case where the Reed-Solomon (RS) code is used as the error correction code. In FIG. 15 reference numeral 78a denotes synchronizing information to establish synchronization of the error correction frame 77, 78b M bytes code word (RS (M, M- N) code word), 79a an information field which stores N bytes information symbols which is protected by the error correction coding, and 79b a check field which stores check symbols which is added to detect error detection.
A channel multiplexing method for the P channels in the transmission processing section 93 will be described with reference to FIG. 16. Each channel corresponds to each program in television broadcasting for example. A channel multiplexing frame 90 includes synchronizing information 90a to establish synchronization of the channel multiplexing frame 90. The channel multiplexing is executed by using the structure multiplexing system.
A transmission frame will be described with reference to FIG. 17. The transmission frame is defined in dependence on the characteristics of the transmission line. Various transmission frames are defined in accordance with the respective types such as the cable transmission, satellite wave transmission and ground wave transmission and so on. FIG. 17 shows one embodiment of a structure of a transmission frame. As shown in FIG. 17, a transmission frame 85 is composed of an additional information area 86 and a transmission information area (payload) 87. Timing information, synchronizing information and information that is used for securing the transmission line, or the like are contained in the additional information area 86. The information that forms the transmission frame 85 is read out in a desired order shown with the arrows in FIG. 17. After the transmission frame 85 was modulated, it is sent to a transmission line as a transmission signal 141.
Next, operations on the receive side of the multimedia information processing system will be described. The operations on the receive side is reverse to those on the transmission side, which were previously described. Namely, after the receive processing section 96 demodulated the received transmission signal 141, it establishes the synchronization of the transmission frame 85 and reproduces, using the synchronizing information in the additional information area 86, and takes out the transmission information area 87. In a case where a signal in the transmission information area 87 is channel multiplexed, the receive processing section 96 establishes the synchronization of the channel multiplexing frame 90 and reproduces, using the synchronizing information 90a, and separates the respective channel information. When each channel information is an error correction coded signal, the receive processing section 96 establishes synchronization of the error correction frame 77 and reproduces, using the synchronizing information 78a. After that, the receive processing section 96 detects errors and correct them.
Descramble processing in the receive processing section 96 will be described using FIG. 14(b). In FIG. 14(b), reference numeral 73b denotes a pseudo random number generator for descramble and 75 an exclusive-OR circuit. The pseudo random number generator 73b to which an initial value 72b is given generates a pseudo random number sequence 74b. The pseudo random number generator 73b in a descramble circuit has the same structure as the pseudo random number generator 73a shown in FIG. 14(a). The initial value 72a is the same as the initial value 72b. The value of the pseudo random number sequence 74a coincides with that of the pseudo random number sequence 74b.
The media demultiplexing section 95 separates the multiplexed bits sequence 122 into the video coded bits sequence 111, one or more audio coded bits sequences 112 and one or more data coded bits sequences 113. The generation of the multiplexing frame 80 necessary for the separation is executed by using synchronizing information contained in the synchronizing information area 81 within the multiplexing frame 80.
In the information-source decoding section 94, The video coded bits sequence 111, audio coded bits sequence 112, and data coded bits sequence 113 are respectively decoded through information-source decoding or various reverse processing by the video decoding section 514, audio decoding section 515 and data decoding section 516 to reproduce the video signal 101, the audio signal 102 relating to the video signal 101 and the data signal 103 relating to the video signal 101 or audio signal 102.
Since a conventional multimedia information processing system is constructed as described above, the processing contents in each processing section and the interface signal between the respective processing sections are fixedly indexed. Consequently, the conventional multimedia information lacks wide usableness.
Therefore, it is difficult to execute data exchange service among broadcasting system, communication system and storage system in computer. Further, it has been difficult to build hardware and add processing functions while maintaining interchangeable properties with other multimedia information processing systems.
Since, in the conventional multimedia information processing system, the video signal, audio signal, and other data have been structure-multiplexed by storing them in given areas in multiplexing frames, multiplexing processing difficult to deal with change in the transmission speed of information to be multiplexed from the respective media and existence of information from the respective media. Further, in a case where the number of media are large, the size of the multiplexing frame must be enlarged. As a result, long time has been required for establishing synchronization of the multiplexing frame on the receiver side.
In case the transmission line is a general line and the transmission signal is a modulated wave by Quadrature Amplitude Modulation (QAM) or Phase Shift Keying (PSK) for example, functions of the transmission processing section 93 and the receive processing section 96 are realized as shown in FIG. 13. However, in case the transmission line is an ATM network, the transmission processing section 93 and the receive processing section 96 should perform ATM physical layer process, ATM layer process, and ATM adaptation layer process as shown in FIG. 18. In case contents of media are exchanged between a multimedia information processing system connected to an ATM network and another multimedia information processing system connected to a general transmission line, adapters are required. The adapters must include both functions shown in FIGS. 13 and 18 if multiplexed bits streams in both systems are same.